report  documentation  page 


form  Approved 
QMS  No.  070A-0rS8 


C  Ti (nM'jC;:On^.  ica^c^ln5 

agssassaaMigggg^^ 

1° ^  ^  - h.  report  type  and  dates  COVERtU 

1.  agency  use  CNL/'  (Eejve  6.0. -v/  j-. 


;  e«'it:r.c 
any  Other 

uC  rC5C3. 


sata  vourcM. 
asoect  o* 

2:S  Jetterson 


i  H.sSvJiv.Su.:ci;CJ.-rli^-9'-Cn.  V^  .  ire  s  ■  t  - - - - REPORT  TYPE 

— AGENCY  use  CNlV  |  i- 

^roc^g<  (50Ov|lf/  •PnAlvt>'i3  jQrti\ll3  5 

:  'I 'i o  fT  6’og3  _  _ _ _ _ 

6.  aUTHOR(5) 

beYE-f  - — — - - - 

7.  performing  organization  NAiME(S)  AND  ADDrvLSS(.S) 

AFIT  Students  Attending: 


DEPRTMENT  OF  THE  AIR  FORCE 
AFIT/ Cl 

2950  P  STREET  | 

WRIGHT-PATTERSON  AFB  OH  45433-7765 


I  8.  PERFORMING  ORGANIZATION 
REPORT  NUMBER 
AFIT/CI/CIA 


10  SPONSORING /monitoring” 
.  agency  REPORT  NUMBER 


I  11.  SUPPLEMENTARY  NOTES 


”23.  distribution /AVAILABILITY  STATEMENT 

Approved  for  Public  Release  lAW  190-1 
Distribution  Unlimited 
MICHAEL  M.  BRICKER,  SMSgt ,  USAF 
Chief  Administration 


1  1 

V  )■> 

' '  1 » 

0 

;V--» 

\r' 

Vj  K-j 

■  w 

1  12b.  DISTRIBUTION  CODE 


I  13.  ABSTRACT  (Maximum  200  wordsf 


19950103  039 


U.  SUBJECT  TERMS 


16.  PRICE  CODE 


An  Tinngft  Quafity  Analy^  of  ANVlS-6  Nig^  Vision  Goggles 


By 

Derek  H.  Abel 

Committee  Chairman:  Robert  J.  Beaton 
Industrial  and  Systems  Engineering 


(ABSTRACT) 


This  study  was  undertaken  in  an  effort  to  relate  ANVIS-6  Night  Vision  Goggle 
image  quality  to  user  performance.  The  purpose  was  to  determine  which  of  five  image 
quality  metrics  best  related  to  performance  tasks.  The  image  quality  metrics  examined 
Modulation  Transfer  Function  Area  (MTFA),  Integrated  Contrast  Sensitivity  (ICS), 
Square  Root  Integral  (SQRI),  Resolution,  and  Signal-to-Noise  Ratio  (SNR).  The 
performance  tasks  were  detection  and  recognition  of  targets  under  various  levels  of 
moon  illumination.  The  metric  that  best  related  to  target  detection  was  SNR.  The 
SNR  results  are  consistent  with  visual  psychophysics  and  SNR  effects.  The  metric  that 
best  related  to  target  recognition  was  Resolution.  The  resolution  results  are  consistent 
with  the  position  that  recognition  performance  improves  for  suprathreshold  targets  as 
resolving  power  increases. 


Accesion  For 

NTIS  CRA&I 
OTIC  TAB 
Unannounced 
Justification 

□ 

By _ 

Distribution  /  1 

Availabili 

y  Codes 

Dist 

Avail 

Sp: 

r.d/O,-  . .  j 

An  Image  Quality  Analysis  of  ANVIS-6  Night  Vision  Goggles 

By 

Derek  H.  Abel 

Thesis  submitted  to  the  Faculty  of  the 
Virginia  Polytechnic  Institute  and  State  University 
in  partial  fulfillment  for  the  degree  of 
Masters  of  Science 
in 

Industrial  and  Systems  Engineering 
APPROVED: 


- 7 - ^ - 7 - 

Dr.  Robert  J.  Beaton,  Chairman 


December  1994 
Blacksburg  Va. 


ACKNOWLEDGMENTS 


This  research  was  made  possible  through  the  cooperation  of  ITT  and  their 
Roanoke  Division.  I  also  wish  to  thank  the  Displays  and  Controls  Lab  at  Virginia 
Tech.  Drs.  R.  J.  Beaton  and  Harry  L.  Snyder  for  all  their  direction  and  advice.  Mr. 
Mike  Snow  and  Mr.  Chuck  Green  who  helped  me  throughout  this  project.  I  especially 
wish  to  thank  Mr.  Woody  Farley,  without  whom  I  would  still  be  in  the  measurements 
laboratory  trying  to  figure  out  how  to  turn  on  the  AEROTECH  positioning  system.  I 
also  wish  to  thank  Tony  Pesante  for  his  advice  on  statistical  analysis  and  being  a  good 
friend. 

Last,  but  not  least,  I  wish  to  thank  my  wife,  Diane,  and  my  sons,  Michael  and 
Anthony,  for  putting  up  with  me  and  supporting  me  throughout  my  days  at  Virginia 


Tech. 


Table  of  Contents 


Abstract 

ii 

Acknowledgments 

iii 

List  of  Figures 

vi 

List  of  Tables 

vii 

INTRODUCTION 

1 

PURPOSE 

10 

LITERATURE  REVIEW 

11 

IMAGE  QUALITY  METRICS 

19 

Modulation  Transfer  Function  Area 

19 

Integrated  Contrast  Sensitivity 

23 

Square  Root  Integral 

24 

Resolution 

25 

Signal-to-Noise  Ratio 

25 

METHOD 

27 

Participants 

27 

Equipment 

28 

Testing  Method 

29 

IV 


RESULTS 


31 


MTFA  vs.  Target  Detection  Performance  33 

ICS  vs.  Target  Detection  Performance  33 

SQRI  vs.  Target  Detection  Performance  36 

Resolution  vs.  Target  Detection  Performance  36 

SNR  vs.  Target  Detection  Performance  36 

MTFA  vs.  Target  Recognition  Performance  40 

ICS  vs.  Target  Recognition  Performance  40 

SQRI  vs.  Target  Recognition  Performance  40 

Resolution  vs.  Target  Recognition  Performance  44 

SNR  vs.  Target  Recognition  Performance  44 

DISCUSSION  47 

CONCLUSION  52 

REFERENCES  53 

APPENDIX  A  58 

APPENDIX  B  59 

APPENDIX  C  60 

APPENDIX  D  61 

VITA  62 


V 


List  of  Figures 


Figure  1.  ANVIS-6  NVGs  (Biberman  and  AUuisi,  1992)  2 

Figure  2.  Image  Intensifier  Tube  (Biberman  and  AUuisi,  1992)  4 

Figure  3.  Multiplier  Tube  and  Typical  Trajectories  (Biberman  and  AUuisi,  1992)  6 
Figure  4.  Measured  Resolution  of  NVGs  (Armentrout,  1993)  15 

Figure  5.  Target  Silhouettes  (Pierce,  1994)  17 

Figure  6.  The  MTFA  Concept  (Snyder,  1985)  20 

Figure  7.  Theoretical  NVG  Resolution  (Armentrout,  1993)  26 

Figure  8.  MTFA  Detection  Regression  Plot  34 

Figure  9.  ICS  Detection  Regression  Plot  35 

Figure  10.  SQRI  Detection  Regression  Plot  37 

Figure  1 1 .  Resolution  Detection  Regression  Plot  38 

Figure  12.  SNR  Detection  Regression  Plot  39 

Figure  13.  MTFA  Recognition  Regression  Plot  41 

Figure  14.  ICS  Recognition  Regression  Plot  42 

Figure  15.  SQRI  Recognition  Regression  Plot  43 

Figure  16.  Resolution  Recognition  Regression  Plot  45 

Figure  17.  SNR  Recognition  Regression  Plot  46 

Figure  18.  MTF  Plots  51 


VI 


List  of  Tables 


Table  1.  Mean  Displacement  Error,  in  Centimeters,  From  Armentrout  (1993)  14 

Table  2.  Percent  Correct  Responses  vs.  Illumination  from  Pierce  (1994)  18 

Table  3.  Results  of  Metric  Calculations  32 

Table  4.  T-test  Results  for  R^  Differences  48 

Table  5.  T-test  Results  for  Correlation  Equal  to  Zero  49 


vii 


INTRODUCTION 


Since  the  early  1970s,  the  frequency  of  use  of  Night  Vision  Goggles  has 
increased  dramatically  in  military  aviation.  Once  the  purview  of  Special  Operations 
and  Special  Forces  units,  the  use  of  ANVIS-6  Night  Vision  Goggles  (NVGs)  (Figure  1) 
has  spread  to  conventional  tactical  aircraft.  Along  with  the  increased  use  of  NVGs, 
there  has  been  an  increase  in  aviation  accidents  involving  NVGs  (Boyd,  1991). 

In  a  study  by  Verona  (1988),  several  aviation  accidents  were  attributed  to  NVGs. 
Some  accident  causes  cited  by  Verona  were  a  lack  of  visual  cues  and  contrast,  flying 
too  fast  for  the  visual  conditions  present,  an  inability  to  determine  distances  to 
obstructions,  and  the  nonvisibility  of  power  lines  and  radio  tower  guide  wires. 

Similarly,  Boyd  (1991)  conducted  a  review  of  Army  rotary-wing  aircraft  accidents 
ranging  in  severity  from  Class-A,  (i.e.,  a  total  loss  of  aircraft  or  a  loss  of  life)  to  Class- 
C,  (i.e.,  repairable  damage  to  equipment  exceeding  $10,000  or  loss  of  work  time) 
between  fiscal  years  1984  and  1989.  During  this  time,  there  were  626  accidents,  of 
which  23  percent  occurred  at  night;  82  percent  of  these  night  accidents  were  attributed 
to  crew  error  with  70  percent  occurring  while  NVGs  were  in  use.  One  hundred  ninety- 
nine  of  all  the  accidents  resulted  in  fatalities.  Forty-one  percent  of  the  fatal  accidents 
occurred  at  night,  with  85%  of  this  number  attributed  to  crew  error.  Seventy-one 
percent  of  the  fatal  crew  error  accidents  occurred  while  NVGs  were  in  use.  The  high 
percentage  of  crew  error  accidents  during  NVG  use  appears  to  point  to  a  breakdown  in 
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Figure  1.  ANVIS-6  NVGs. 

(  Source:  Biberman  and  Alluisi,  1992) 
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the  safe  use  of  NVGs  or  a  deficiency  of  the  NVG  system  itself  (Smith  and  Fedor, 
1984).  It  should  be  noted  that  one  must  be  cautious  when  accepting  the  term  "crew 
error,"  because  it  is  used  often  by  investigators  as  a  catch-all  phrase  to  explain 
accidents  when  exact  causes  are  unclear  or  unknown. 

In  an  attempt  to  reduce  the  number  of  NVG  related  accidents,  one  would  have  to 
look  at  two  possible  avenues.  The  first  approach  is  an  improvement  of  the  system, 
while  the  second  is  training  in  the  safe  use  of  the  NVGs.  System  improvement  and  safe 
employment  of  the  devices  depend  on  a  better  understanding  of  the  processes  that  occur 
when  looking  through  NVGs.  It  also  is  important  to  note  that  while  NVGs  can 
enhance  night  flying  capabilities,  they  do  not  turn  night  into  day  (Armentrout,  1993). 

According  to  Biberman  and  Alluisi  (1992),  NVGs  can  be  broken  down  into  four 
basic  components: 

1 .  The  mounting  frame  which  holds  all  the  components, 

2.  An  objective  lens  to  focus  the  image  onto  the  photocathode  image  intensifier, 

3.  A  photocathode  image  intensifier,  and 

4.  A  unity  magnification  eyepiece  that  can  be  focused  by  the  observer. 

The  photocathode  image  intensifier  section  consists  of  four  distinct  parts:  a 
photocathode,  a  microchannel  plate  (MCP),  a  phosphor  screen,  and  a  power  supply 
(Figure  2).  The  photocathode  receives  photons  either  directly  from  a  source  or 
reflected  from  an  object,  whether  the  object  is  natural  or  man-made,  through  the 
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OTOCATHODE 


Figure  2.  Image  intensifier  tube. 

(  Source:  Biberman  and  Alluisi,  1992) 


objective  lens.  Each  photon  striking  the  front  of  the  photocathode  releases  a 
corresponding  electron  on  the  reverse  side.  The  photoelectrons  then  strike  the  MCP 
where  intensification  occurs.  The  MCP  is  composed  of  millions  of  channel  multipliers 
formed  by  stretching  and  fusing  optical  fibers.  Each  optical  fiber  is  a  hollow  glass  tube 
with  lead  coating  on  the  inside  surface.  Whenever  a  photoelectron  strikes  the  side  of 
the  fiber  optic  tube,  it  releases  electrons  from  the  lead  coating.  By  running  an 
electrical  current  through  the  coating,  making  electrons  available,  the  channel  achieves 
the  function  of  multiplying  each  photoelectron  via  a  chain  reaction  (Figure  3).  Within 

the  MCP,  each  photoelectron  produces  on  the  order  of  lO^  to  10^  secondary  electrons. 
The  stream  of  electrons  emitted  from  the  back  of  the  MCP  is  projected  onto  a  phosphor 
screen,  creating  the  monochrome  image  that  is  viewed  through  an  eye  piece.  The  eye 
piece  and  the  objective  lenses  are  conventional  unity  magnification  optics. 

NVGs  have  a  number  of  limitations  that  must  be  recognized  before  being  used 
safely.  The  four  most  serious  limitations  are  discussed  below. 

The  first  limitation  is  that  NVGs  permit  at  best  a  visual  acuity  of  20/40  on  the 
Snellen  scale  (Price  and  McLean,  1985).  This  acuity  can  only  be  reached  if  the 
goggles  are  focused  correctly.  Biberman  and  AUuisi  (1992)  showed  that  the  20/40 
acuity  was  achieved  rarely  by  air  crew  members. 

The  second  and  third  limitations  are  physical  in  nature.  The  second  limitation  is 
the  field  of  view  of  the  goggles,  which  is  40  degrees.  The  field  of  view  severely  limits 
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Figure  3.  Multiplier  tube  and  typical  trajectories, 
(  Source:  Biberman  and  Alluisi,  1992) 
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the  aviator's  peripheral  vision.  This  limitation  has  been  likened  by  many  aviators  to 
flying  while  looking  through  two  toilet  paper  rolls. 

The  third  limitation  is  visual  noise.  Noise  appears  in  two  forms:  image  noise 
and  goggle-induced  noise.  Goggle  induced  noise  consists  of  three  effects:  "sparkle," 
"blooming,"  and  "halo."  Sparkle  is  caused  by  random  photoelectrons  striking  the 
photocathode,  it  also  is  referred  to  as  "snow"  since  it  resembles  snow  falling  in  front  of 
your  car  while  your  headlights  are  on.  Blooming  and  halo  are  related  noise  effects; 
both  occur  when  bright  lights  are  viewed  through  the  goggles.  Blooming,  also  referred 
to  as  "washing  out,"  is  caused  by  the  spread  of  intensified  light  around  the  edge  of  the 
image  on  the  screen,  resulting  in  a  blurred  image.  Halo  is  a  brighter  ring  around  an 
image  that  could  cause  a  less  bright  image  to  go  unnoticed. 

The  fourth  limitation  is  degraded  depth  perception  while  wearing  NVGs.  This 
limitation  was  studied  by  Armentrout  (1993).  Armentrout  (1993)  found  that  depth 
perception  performance  decreases  after  ambient  light  is  raised  above  50%  moon 
illumination.  This  also  is  borne  out  when  speaking  to  aviators  that  use  the  NVGs  on  a 
daily  basis;  however,  the  preferred  illumination  levels  vary  from  50%  to  70%  moon 
illumination  (Personal  communication  with  U.S.  Air  Force  and  U.S.  Army  aircrew 
members  from  1989  through  1993). 

These  limitations  highlight  NVG  problems  that  need  further  investigation.  The 
first  is  that  a  great  number  of  aviators  are  very  comfortable  when  flying  on  high 
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illummation  nights  (Personal  communication  with  U.S.  Air  Force  and  U.S.  Army 
aircrew  members  from  1989  through  1993).  Studies  by  Armentrout  (1993)  and 
Riegler,  Whitely,  Task,  and  Schueren  (1991)  suggest  that  as  illumination  levels 
increase  above  50%  moon  illumination,  there  is  a  decrease  in  the  resolving  power  and 
signal-to-noise  ratio  of  the  NVGs  that  degrades  performance.  This  performance 
degradation  at  the  higher  illuminations  could  be  leading  to  an  unwarranted  sense  of 
comfort  and  complacency  while  flying.  The  current  thinking  in  the  U.S.  Air  Force  and 
the  U.S.  Army  is  that  higher  illumination  levels  lead  to  better  pilot  performance. 

In  this  paper,  five  image  quality  metrics  will  be  considered  to  see  how  they  relate 
to  performance  of  persons  using  ANVIS-6  NVGs.  The  metrics  considered  are:  Signal- 
to-Noise  Ratio  (SNR),  Resolution,  Modulation  Transfer  Function  Area  (MTFA), 
Integrated  Contrast  Sensitivity  (ICS),  and  SQuare  Root  Integral  (SQRI).  There  are 
more  image  quality  metrics  available  for  analyzing  system  performance,  many  of  which 
are  beyond  the  scope  of  this  paper  (Beaton,  1984). 

Studies  on  SNR  effects  in  visual  psychophysics  can  be  traced  back  to  the  work  of 
Fechner  (1860  cited  by  Gescheider,  1985:  Hecht,  Shlaer,  and  Pirenne,  1942;  Rose 
1942;  and  de  Vries,  1943,  cited  by  Schnitzler,  1973).  Since  this  early  work,  SNR 
effects  have  been  studied  in  the  context  of  imaging  devices  with  varying  degrees  of 
success  (Task,  1979,  Beaton,  1984). 

Resolution  is  a  system  parameter  that  is  related  directly  to  the  edge  sharpness  on 
a  display.  It  also  is  related  to  display  quality  in  that  the  smaller  the  discernible  details 
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on  the  display,  the  higher  the  quality  of  the  display.  Because  resolution  is  a  system 
parameter,  one  would  expect  it  to  be  constant  for  a  given  device.  Armentrout  (1993) 
found  that  for  ANVIS-6  NVGs,  resolution  was  not  constant  across  different  moon 
illumination  levels. 

The  concept  of  Modulation  Transfer  Function  Area  (MTFA)  was  proposed  first 
by  Charman  and  Olin  (1965,  as  reported  by  Snyder,  1973)  as  Threshold  Quality  Factor 
(TQF).  TQF  was  renamed  MTFA  by  Borough,  H.C.,  Fallis,  R.F.,  Wamock,  R.H., 
and  Britt,  J.H.  (1967,  cited  by  Snyder,  1973).  Charman  and  Olin  (1965,  as  reported 
by  Snyder,  1973),  characterized  MTFA  as  a  measure  of  photographic  image  quality 
that  contains  the  cumulative  effect  of  the  various  stages  of  the  observation  process,  the 
"noise"  introduced  into  the  perceived  image,  and  the  limitations  imposed  by  the 
physiological  and  psychological  characteristics  of  the  observer.  In  two 
photointerpretation  studies  cited  by  Snyder  (1973),  Klingberg  (1970)  and  Borough  et 
al.,  (1967),  MTFA  was  found  to  have  correlation  coefficients  of  0.92  and  -0.93, 
respectively  (the  negative  value  was  due  to  the  use  of  number  of  errors  as  a  human 
performance  measure,  which  was  inversely  related  to  MTFA). 

Integrated  Contrast  Sensitivity  (ICS)  was  introduced  by  Van  Meeteren  (1973)  as 
an  improvement  over  MTFA.  Since  multiplication,  rather  than  subtraction,  is  the 
integrand  operator  involved  in  the  ICS  equation,  it  is  more  responsive  to  small  changes 
in  either  the  system  Modulation  Transfer  Function  (MTF)  or  the  visual  Contrast 
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Threshold  Function  (CTF)  than  the  MTFA  metric.  Task  (1979)  found  that  ICS  had  a 
higher  correlation  with  performance  than  did  the  MTFA. 

The  SQRI  metric,  proposed  by  Barten  (1987),  is  another  MTF-based  metric.  The 
SQRI  metric  uses  non-linear  scaling  of  modulation  across  spatial  frequencies.  This 
non-linear  scaling  matches  observed  patterns  in  human  suprathreshold  contrast 
discriminations.  It  does  so  by  giving  greater  weight  to  the  contributions  of  lower 
modulation  levels  at  lower  spatial  frequencies  through  a  square  root  operator  and 
logarithmic  integration. 

While  a  number  of  studies  have  examined  individual  image  quality  effects  in 
relation  to  NVGs  (Armentrout,  1993;  Riegler  et  al.,  1991),  none  have  examined 
several  image  quality  metrics  m  the  same  study.  Since  improved  image  quality  should 
allow  the  pilot  to  see  obstacles  better,  it  is  important  to  ascertain  which  metric  best 
relates  to  performance  so  engineers  can  design  safer  NVGs. 

PURPOSE 

The  purpose  of  this  project  was  to  evaluate  image  quality  metrics  for  NVGs. 

This  work  assessed  the  relationships  between  MTFA,  SNR,  Resolution,  ICS,  and  SQRI 
image  quality  metrics  with  two  human  visual  performance  measures. 
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LITERATURE  REVIEW 


A  study  conducted  by  Riegler,  Whitely,  Task,  and  Schueren  (1991)  examined  the 
effect  of  SNR  on  visual  acuity.  Visual  acuity  was  measured  by  determining  the 
minimum  angular  subtense  of  a  test  character  (e.g.,  Landolt  "C,"  tumbling  "E,"  or 
Snellen  letter).  The  observers  in  this  study  were  asked  to  determine  the  orientation  of 
the  Landolt  "C"  or  the  tumbling  "E"  or  to  read  characters  on  a  Snellen  chart.  Visual 
acuity  was  measured  at  two  contrast  levels  (20%  and  95%),  two  illumination  levels 
(25%  and  1%  lunar  disk),  and  four  SNRs  (17.92,  15.28,  13.71,  and  11.37).  At  the 
beginning  of  each  trial,  observers  were  seated  in  a  golf  cart  12.2  m  from  the  acuity 
chart.  The  cart  then  was  moved  toward  the  chart  and  stopped  by  the  subject  when  he 
was  "virtually  certain"  that  he  could  determine  the  orientation  of  all  the  "C"s  on  the 
Landolt  chart.  If  the  response  was  incorrect,  the  cart  was  restarted  and  it  proceeded 
down  the  track  until  the  subject  could  identify  the  orientation  or  until  the  end  of  the 
track.  Visual  acuity  was  measured  as  the  minimum  angle  of  resolution  computed  using 
the  distance  from  the  visual  acuity  targets  when  the  subjects  correctly  identified  the 
orientation  of  all  Landolt  "C"s. 

The  results  of  this  experiment  showed  that  the  SNR  does  affect  visual  acuity 
through  the  NVGs  at  low  levels  of  moon  illumination.  Specifically,  increases  in 
intensifier  SNR  resulted  in  better  visual  acuity  at  both  quarter  moon  and  starlight  (1  % 
moon)  illumination  levels  and  at  high  and  low  target  contrasts. 
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Armentrout  (1993)  examined  depth  perception  while  looking  through  NVGs.  A 
modified  Howard-Dolman  apparatus  was  used  under  the  levels  of  illumination  and 
contrast  described  earlier.  The  participants  (i.e.,  10  male  and  2  female  volimteers) 
were  screened  for  far  distance  visual  acuity  and  stereoacuity  using  a  Bausch  and  Lomb 
Ortho-Rater.  The  volunteers  were  required  to  meet  a  minimum  criterion  of  20/25 
Snellen  acuity  and  at  least  marginal  stereoacuity  (top  two  lines  on  the  test  screen). 

The  bars  of  the  Howard-Dolman  apparatus  were  replaced  by  flat  targets  made  of 
exposed  photographic  paper.  The  target  bars  were  cut  from  paper  with  differing 
exposures  to  provide  three  distinct  contrasts.  The  contrasts  were  83%,  53%,  and  25%, 
using  positive  target-to-background  calculations.  The  contrast  conditions  were 
measured  with  a  Minolta  CS-100  chromometer. 

The  targets  were  illuminated  from  a  single  calibrated  light  source,  and  neutral 
density  filters  were  used  to  manipulate  the  illumination  levels.  The  levels  of 
illumination  used  by  Armentrout  were  full  moon,  half  moon,  10  percent  moon  and  1 
percent  moon  (star  light)  illumination.  These  equated  to  0.2152,  0.1076,  0.02152,  and 
0.002152  lux,  respectively  (RCA,  1974).  The  illumination  measures  were  performed 
using  a  Minolta  T-1  illuminance  meter. 

The  participant's  task  was  to  align  the  right  target  rod  until  it  appeared  to  be  in 
the  same  plane  as  the  fixed,  left  target  rod.  The  starting  position  for  the  right  target 
fell  randomly  into  1  of  10  locations  fore  or  aft  of  the  stationary  left  target  {+ 1- \ 
through  +/-  10  centimeters).  When  the  participant  was  sure  the  targets  were  aligned. 
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the  linear  displacement  of  the  right  target  was  measured.  The  mean  displacement  error 
(performance  variable)  for  each  treatment  combination  (Table  1)  was  the  dependent 
variable. 

The  results  of  the  study  indicated  that  stereoacuity  was  worse  at  50%  and  100% 
moon  illumination;  in  other  words,  performance  increased  to  a  point  with  increased 
illumination  and  then  decreased  as  illumination  increased  further.  Armentrout  related 
this  performance  decrement  to  a  decrease  in  resolution  at  the  higher  illuminations 
(Figure  4),  suggesting  that  the  resolution  decrement  comes  about  due  to  halo  and 
blooming  display  effects  at  the  higher  levels. 

The  two  performance  measures  used  in  the  present  work  came  from  an 
experiment  conducted  by  Pierce  (1994).  In  Pierce's  study,  participants  were  asked  to 
detect  and  recognize  targets  under  various  illumination  levels,  target  sizes,  and  target 
contrasts.  Specifically,  there  were  four  levels  of  illumination:  full  moon  (0.2152  lux), 
three-quarter  moon  (0.1614  lux),  half  moon  (0.1076  lux),  and  one-quarter  moon 
(0.0538  lux).  The  three  levels  of  positive  target-to-background  contrast  used  were 
86.86%,  53.63%,  and  17.58%.  The  third  independent  variable  in  Pierce's  study  was 
target  size,  which  had  four  levels:  8.75,  17.5,  35.0,  and  70.0  minutes  of  visual  angle. 
For  the  experiment,  the  participants  were  placed  in  a  darkened  room,  1.8  meters  away 
from  a  rear  projection  screen,  and  asked  to  look  through  a  pair  of  ANVIS-6  NVGs  at 
the  screen.  Using  one  of  two  slide  projectors,  targets  were  shown  on  the  screen. 
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TABLE  1.  Mean  Displacement  Error,  in  Centimeters,  from  Armentrout  (1993) 
Means  for  Viewing  Condition. 


Condition 

Mean 

Std.  Dev, 

Day 

3.592 

2.572 

Night 

lAll 

3.863 

Means  for  Viewing  Condition  vs.  Illumination 


Condition 

Mean 

Std.  Dev. 

Day,  100 

3.183 

2.459 

Day,  50 

3.773 

2.827 

Day,  10 

3.454 

2.722 

Day,  1 

3.959 

2.282 

Night,  100%  Moon 

8.064 

4.897 

Night  50%  Moon 

8.326 

4.388 

Night  10%  Moon 

6.442 

2.275 

Night  1  %  Moon 

7.074 

3.057 
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Log  Illumination  (fC) 

Measured  resolution  Theoretical  resolution 


Figure  4.  Measured  resolution  of  NVGs. 
(Source:  Armentrout,  1993) 
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The  targets  were  silhouettes  of  a  tank,  armored  personnel  carrier  (APC),  and  a  civilian 
car  (see  Figure  5).  The  targets  also  were  shown  individually  at  random  locations  on 
the  screen.  In  relation  to  a  clock,  the  target  positions  were  12:(X),  1:30,  3:00,  4:30, 
6:(X),  7:30,  9:(X),  and  10:30.  On  each  trial,  the  participant  was  asked  to  say  the 
location  of  the  target  and  then  to  state  the  type  of  target. 

The  two  dependent  variables  were:  50%  target  detection  probability  and  50% 
target  recognition  probability.  Data  analysis  consisted  of  reducing  the  data  into  the 
probabilities  of  target  detection  and  then  target  recognition  for  each  viewing  condition. 
The  probability  of  correct  target  detection  was  calculated  by  taking  the  number  of 
correct  responses  given  (i.e.,  the  target  was  said  to  be  at  12:(X)  when  it  actually  was  at 
12:00)  in  each  treatment  combination  divided  by  the  total  number  of  trials  for  each 
treatment  combination.  The  probability  of  correct  target  recognition  under  each 
viewing  condition  was  calculated  in  almost  the  same  way,  the  difference  being  correct 
recognition  was  only  counted  if  the  target  had  been  successfully  detected.  The 
percentages  of  correct  responses  for  target  detection  and  target  recognition  for  Pierce’s 
four  illuminations  are  shown  in  Table  2.  After  the  initial  data  reduction,  the  average 
probabilities,  collapsed  across  replications  and  target  type  for  each  observer,  were 
subjected  to  Analysis  of  Variance  (ANOVA)  procedures.  Pierce  (1994)  found  the 
effects  of  illumination,  contrast,  and  target  size  to  be  significant. 
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Tank 

nixistration  of  target  types 


Figure  5.  Target  silhouettes. 
(Source:  Pierce,  1994) 
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TABLE  2.  Percent  Correct  Responses  vs.  Illumination  from  Pierce  (1994) 


niumination 

Detection 

Recognition 

100%  Moon 

58.18% 

45.21% 

75%  Moon 

54.32% 

41.43% 

50%  Moon 

51.18% 

34.58% 

25%  Moon 

44.36% 

34,01% 
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Another  study  investigating  the  relationship  of  visual  acuity  and  NVG  use  was 
conducted  by  Wiley  (1989).  In  this  study,  the  effects  of  contrast  and  luminance  on 
visual  acuity  were  considered  under  six  viewing  conditions.  Size  and  contrast  were 
manipulated  in  this  study.  The  size  of  the  target,  a  Snellen  "E,"  varied  from  20/10  to 
20/400,  in  Snellen  notation,  and  contrast  was  set  at  5,  35,  and  94  percent.  The  targets 
were  presented  in  one  of  four  orientations  and  the  observers  were  asked  to  determine 
the  orientation  of  the  "E"  on  each  trial.  Wiley  found  that  as  luminance  and  contrast 
were  reduced,  visual  acuity  decreased.  This  leads  one  to  believe  that  luminance  and 
contrast  impact  visual  acuity  with  NVGs.  The  findings  of  Barlow  (1958)  and 
Blackwell  (1946),  among  others,  lead  to  the  same  conclusion.  Barlow  (1958)  and 
Blackwell  (1946)  stated  that  as  luminance  levels  increase,  the  contrast  required  to  detect 
a  target  decreases. 


IMAGE  QUALITY  METRICS 
Modulation  Transfer  Function  Area 

This  metric  takes  into  account  the  MTF  of  an  imaging  system  as  well  as  the  CTF 
of  the  visual  system  (Snyder,  1985)  when  assessing  image  quality  (Figure  6).  To 
adequately  define  MTFA  we  need  to  first  describe  its  components,  namely  MTF  and 
CTF.  The  MTF  is  the  plot  of  Modulation  Transfer  Factors  across  the  spatial  frequency 
passband  of  an  imaging  system.  By  knowing  the  modulation  of  a  sine  wave  going  into 
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MODULATION  TRANSFER  FACTOR 


SYSTEM  MTF  CURVE 


LIMITING  RESOLUTION 


DETECTION 

THRESHOLD 

CURVE 


SPATIAL  FREQUENCY  - ^ 


Figure  6.  The  MTFA  concept. 
( Source:  Snyder,  1985) 
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a  system,  and  by  measuring  the  modulation  of  the  sine  wave  coming  out  if  the  system, 
one  calculates  the  Modulation  Transfer  Factor  using  the  equation: 

Modulation  Transfer  Factor  =  Mom !  ,  (1) 

where  modulation  (M)  is  defined  as: 

^  (f^Max  “  ^Min  )  ^  (  ^Max  "h  (2) 


and  IvMax  =  maximum  luminance  of  the  lighter  grating  half-cycle,  and 
LMin  ~  minimum  luminance  of  the  darker  grating  half-cycle. 

Linear  systems  analysis  makes  it  possible  for  one  to  determine  the  transmission 
capacity  of  a  given  system  across  its  spatial  firequency  passband.  The  Fourier  Theorem 
states  that  any  waveform  can  be  broken  down  into  a  series  of  sine  waves,  each  having  a 
unique  amplitude  and  phase  relationship  to  each  other. 

The  CTF  is  the  modulation  that  must  be  received  by  the  human  eye  to  detect  a 
sine  wave  pattern  at  a  given  frequency.  The  CTF  can  be  determined  by  psychophysical 
experiments  or  calculated  using  an  equation  proposed  by  Barten  (1990). 

The  equation  proposed  by  Barten  (1990)  is: 


(-bu)  (bu)  1/2 

CTF  =  1/au  [  1  +  c  ] 


(3) 


540 

where  a  — - 

DW  (l 

b  =  0.3  (1  +  100/L)  , 
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c  =  0.06, 


u  =  spatial  frequency  in  cycles  per  degree, 

DW  =  the  angular  display  size  in  degrees  calculated  from  the  square  root  of  the 
picture  area,  and 

L  =  the  effective  adaptive  display  luminance  in  candelas  per  square  meter. 

In  psychophysical  CTF  experiments,  an  observer  adjusts  the  luminance  contrast 
(modulation)  of  the  grating  to  a  point  where  the  sine  wave  pattern  can  just  be  detected. 
The  modulation  needed  to  reach  a  threshold  response  is  an  indication  of  the  sensitivity 
of  the  observer  to  that  spatial  frequency  pattern. 

When  plotting  the  threshold  contrast  as  a  function  of  the  spatial  frequency,  the 
result  is  the  CTF.  The  inverse  of  the  CTF  is  known  as  the  Contrast  Sensitivity 
Function  (CSF). 

MTFA  is  related  to  both  MTF  and  CTF  by  the  equation; 

MTFA-=  f  MTF  (/) -CTF  (/)df  (4) 

0 

or 

fc 

MTFA  =  J  MTF  (/)  -CSF  (5) 
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where  f,. ,  where  f  is  the  spatial  frequency  at  the  eye  of  the  observer  in  cycles/degree 
and  the  subscript,  c,  denotes  the  spatial  frequency  point  at  which  the  MTF  curve 
crosses  the  CTF  curve. 

Thus,  the  MTFA  is  a  measure  of  a  system's  capacity  to  present  detectable  modulation 
across  the  observer's  spatial  frequency  passband. 

Task  (1979)  evaluated  the  MTFA,  along  with  other  image  quality  metrics,  and 
found  high  correlations  between  visual  task  performance  and  the  MTFA.  Similar 
results  were  found  by  Beaton  (1984),  Joma  (1989),  Snyder  (1973),  and  Snyder  and 
Maddox  (1980). 

Integrated  Contrast  Sensitivity  (ICSl 

This  metric  also  is  related  to  the  MTF  and  CTF.  As  stated  earlier,  it  was 
introduced  by  Van  Meeteren  (1973)  as  an  improvement  over  MTFA.  Since 
multiplication,  rather  than  subtraction,  is  the  operator  involved  in  the  ICS  equation,  it 
is  more  responsive  to  small  changes  in  either  the  system’s  MTF  or  the  visual  CTF  than 
is  the  MTFA  metric  (Joma,  1993).  The  equation  for  calculating  ICS  is: 

fiw 

ICS=  f  MTF  (J)l  CTF  {f)df  ,  (6) 

0 

where  f  is  the  spatial  frequency  at  the  eye  of  the  observer  in  cycles/degree,  MTF(f)  is 
the  MTF  of  the  display,  CTF(f)  is  the  CTF  at  the  spatial  frequency,  and  ^ny  is  the 

Nyquist  frequency.  The  Nyquist  frequency  is  the  maximum  spatial  frequency 
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transmitted  by  the  imaging  system  (Dainty  and  Shaw,  1974).  Beaton  (1984)  and  Task 
(1979),  found  a  slightly  higher  numerical  correlation  between  performance  measures 
and  ICS  than  with  the  MTFA  metric. 

Square  Root  Integral  tSORll 

A  third  variant  of  the  MTF-based  metrics  was  proposed  by  Barten  (1987).  It  was 
called  SQRI  and  it  resembled  the  ICS  metric.  The  difference  between  SQRI  and  ICS  is 
that  the  perceptually  weighted  MTF  area  in  SQRI  is  transformed  with  a  power 
function,  followed  by  a  logarithmic  integration.  The  non-linear  behavior  of  the  human 
visual  system  is  accounted  for  by  taking  the  square  root  of  the  ratio  between  the  display 
MTF  and  the  CTF.  The  most  recent  SQRI  metric  equation  (Barten,  1990)  is: 

SQRI  ^  f  [MTFif)  /  CTFif)]  (7) 

In  2  ^3 

where  f  is  the  angular  spatial  frequency  at  the  eye  of  the  observer  in  c/deg;  MTF(f)  is 
the  MTF  of  the  display,  and  CTF(f)  is  the  Contrast  Threshold  Function.  The  lower 
integration  limit,  fj,  is  either  zero  spatial  frequency  or  the  lowest  frequency  that  can  be 
determined  by  the  size  of  the  displayed  image.  The  upper  integration  limit  f2  ,  is  the 
highest  spatial  frequency  that  can  be  determined  by  the  size  of  the  displayed  image. 

The  units  for  SQRI  are  expressed  in  terms  of  Just  Noticeable  Differences  (jnd),  with 
one  jnd  defined  as  a  detection  probability  of  75%  in  a  two-alternative  forced-choice 
experiment. 


24 


Resolution 


According  to  the  ANSI/HFS  100-1988  Standard,  resolution  is  the  ability  of  a 
visual  device  to  display  the  smallest  discernible  details.  This  measure  is  strictly  a 
system  parameter,  referring  to  the  size  of  the  picture  elements  on  the  screen.  In  the 
case  of  ANVIS-6  NVGs,  the  best  available  resolution  is  40  line  pairs  per  millimeter 
(LP/mm).  If  one  doesn't  take  static  noise  such  as  halo  and  blooming  into  account,  then 
resolution  theoretically  remains  at  40  LP/mm.  This  is  not  how  the  NVGs  work, 
however.  As  illumination  decreases,  the  gain  in  the  goggles  automatically  increases, 
resulting  in  more  goggle-induced  noise.  The  increase  in  noise  results  in  a  decrease  in 
resolution.  Resolution  is  measured  by  looking  at  Air  Force  standard  three-bar  chart  to 
determine  the  smallest  spatial  frequency  that  can  be  viewed  as  separate  lines  through  an 
imaging  system.  Theoretical  resolutions  for  different  illuminations  are  shown  in  Figure 
7.  The  limiting  resolution  of  a  system  is  the  spatial  frequency  at  which  an  observer  can 
no  longer  discriminate  the  light  and  dark  bars  of  a  square- wave  image  (Snyder,  1985 
p.  87). 

Signal-To-Noise  Ratio  fSNR) 

Beaton  (1984)  found  that  the  SNR  metric  correlates  well  with  performance  tasks 
in  both  hard-  and  soft-copy  displays.  In  fact,  Beaton  found  that  the  SNR  metric  had 
the  highest  average  R^  (86.8%)  across  four  different  experiments  of  the  16  metrics 
considered.  SNR  is  calculated  by  relating  the  amount  of  signal  energy  to  be  detected 
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Figure  7.  Theoretical  NVG  resolution. 
( Source:  Armentrout,  1993) 
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by  the  participant  against  a  background  energy  (noise).  The  relationship  is  the  amount 
of  signal  energy  divided  by  the  noise  energy.  In  this  case  the  signal  was  the  measured 
DC  current  coming  from  a  Gamma  Scientific  GS-21 10  Telemicroscope  that  was 
pointed  at  the  NVGs.  The  noise  was  the  AC  current  coming  from  a  Gamma  Scientific 
GS-21 10  Telemicroscope  that  was  pointed  at  the  NVGs. 


METHOD 

Task  (1979)  said  that  there  are  two  approaches  to  determining  functional  image 
quality:  subjective  and  objective.  The  subjective  approach  involves  the  comparison  of 
an  analytically  derived  image  quality  measure  and  a  subjective  human  quality 
assessment  of  the  same  image.  The  objective  approach  takes  an  analytically  derived 
measure  of  image  quality,  but  it  compares  it  to  a  performance  variable  associated  with 
a  specific  task.  In  this  study,  the  objective  approach  was  taken. 

Participants 

The  participants  for  the  performance  database  used  in  this  work  (Pierce,  1994) 
were  males  and  females  between  the  ages  of  22  and  34  years.  There  were  12 
participants  in  the  study,  6  male  and  6  female.  The  participants  were  screened  for 
visual  acuity  of  20/20  (corrected)  using  the  Ortho-Rater.  Since  current  Department  of 
Defense  policy  allows  for  women  in  combat,  and  aviators  are  typically  between  the 
ages  of  22  (graduation  from  flight  training)  and  42  (retirement  or  senior,  mission  ready 


27 


only  fliers),  participants  in  the  study  provide  a  close  match  with  the  military  aviation 
population  currently  using  NVGs. 

Equipment 

The  NVGs  used  were  AJNVIS-6  goggles  produced  by  ITT.  The  tubes  used  were 
numbers  34425  (left  tube)  and  37270  (right  tube);  these  were  the  same  tubes  used  by 
Armentrout  (1993)  and  Pierce  (1994). 

The  target  was  illuminated  using  a  Hoffman  Energy  Corp.  Spectral  Radiance 
Standard.  Power  for  the  Spectral  Radiance  Standard  was  provided  by  a  KEPCO  Power 
Supply  (model  ATE  15-6m).  Neutral  density  filters  were  used  to  simulate  the  different 
moon  illumination  levels. 

The  observation  target  was  a  black-on- white  (85%  contrast)  three-bar  chart,  made 
from  matte  board.  The  size  of  the  white  matte  board  was  76.2  cm  x  101.6  cm.  The 
black  target  bars  were  cut  to  produce  1,  2.5,  5,  10,  and  15  cycles  per  degree 
observation  targets. 

The  MTFs  of  the  NVGs  were  calculated  using  the  Questar  2000  OMI 10151  Mk 
in  SZ,  which  was  run  by  PMIS  201  software  on  an  IBM  486/33  MHz  computer.  The 
AEROTECH  Unidex  1 1  positioning  system  was  used  to  place  the  goggles  in  front  of 
the  Questar  for  measurement.  Equations  for  the  Modulation  Transfer  Functions 
(Appendix  A)  were  found  using  DELTA  GRAPH  software  on  a  Macintosh  computer. 
MATHEMAJICA  (version  2.2.2.)  was  used  to  calculate  the  MTFA,  ICS,  and  SQRI. 
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The  SNR  level  was  found  using  a  Gamma  Scientific  GS-2110  Telemicroscope,  a 
Singer  323-OlTrue  RMS  Voltmeter,  Fluke  8000A  Digital  Multimeter,  and  a  Photo 
Research  DR-2  Digital  Radiometer  with  a  1mm  aperture  and  a  lx  magnification  lens. 
The  telemicroscope  was  aimed  at  the  MCP  on  the  NVGs.  The  signal  from  the  Gamma 
Scientific  GS-  2110  Telemicroscope  then  was  passed  to  the  digital  multimeter  and  the 
RMS  voltmeter  through  a  single  cable  with  a  T-connection  at  the  digital  multimeter. 
The  AC  current  was  measured  on  both  the  multimeter  and  the  voltmeter  yielding  the 
same  value  on  both  instruments. 

Resolution  measurements  were  performed  at  ITT  Roanoke  using  a  Hoffman 
Test  Bench.  The  Hoffman  Test  Bench  also  was  used  to  check  the  signal-to-noise 
measurements. 

The  statistical  calculations  and  regression  plots,  were  done  on  STATGRAPHICS 
(version  5). 

Testing  Method 

First,  the  MTFs  were  calculated.  This  was  done  by  placing  a  three-bar  chart 
three  meters  from  the  NVGs.  The  Questar  then  was  focused  on  the  MCP  of  the  NVGs. 
Each  of  five  spatial  frequencies  (i.e.,  1,  2.5,  5,  10,  and  15  cycles  per  degree)  on  the 
target  board  were  illuminated  at  each  of  six  moon  illumination  levels  (i.e.,  100%, 

75%,  50%,  25%,  10%,  and  1%  lunar  disk).  The  PMIS  201  software  was  used  to  plot 
the  maximum  and  minimum  luminance  for  each  spatial  frequency  at  each  illumination 
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level.  The  plots  were  converted  into  McrosoA  EXCEL  (version  5.0)  spreadsheets  so 
the  average  maximum  and  minimum  luminance,  hence  each  Mout  (Eqn.  1),  could  be 
calculated  for  the  spatial  frequencies  at  each  illumination  level.  To  compute  Mi„  (Eqn. 
1)  to  the  NVGs,  the  maximum  and  minimum  luminance  values  of  the  three  bar  target 
were  determined  using  the  same  procedure  with  one  difference;  the  Questar  was 
focused  directly  at  the  chart  instead  of  looking  through  the  NVGs. 

DELTA  GRAPH  was  used  to  determine  the  equations  for  the  MTFs.  The  MTF 
data  were  fit  using  a  Gaussian  MTF  model  and  the  Levinson-Marquardt  method  (Delta 
Point  Inc.,  1993).  With  the  MTF  determined  using  the  Questar,  and  the  CTF  found 
using  Barten’s  Equation,  the  MTFAs,  ICSs,  and  SQRIs  were  calculated 
MATHEMATICA  (version  2.2.2). 

SNRs  were  calculated  from  the  NVGs  viewing  a  white  matte  board  under  the  six 
different  illumination  conditions.  The  Gamma  Scientific  2110  telemicroscope  was  set 
up  to  look  at  the  MCP  of  the  NVGs.  The  signal  coming  out  of  the  telemicroscope  was 
routed  through  the  RMS  voltmeter  and  the  Digitizing  Multimeter.  The  value  of  DC 
divided  by  RMS  volts  was  taken  as  the  SNR.  It  was  assumed  that  DC  current  was  the 
intensity  of  illumination  because  the  power  source  for  the  illuminator  was  constant  DC. 
It  was  also  assumed  that  AC  readings  from  the  True  RMS  Voltmeter  were  noise. 

Resolution  was  measured  at  ITT  Roanoke  using  an  Air  Force  standard  three-bar 
chart  and  a  Hoffman  Test  Bench.  The  resolution  of  each  tube  was  taken  and  the  two 
resolutions  were  averaged  to  get  an  estimate  of  system  resolution. 
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In  this  study,  it  was  decided  to  concentrate  on  the  center  of  the  NVG  operating 
range  and  avoid  the  extreme  illuminations  (1%,  10%,  and  100%  Moon),  because  of  the 
system’s  automatic  gain  control  responses  at  the  lower  and  upper  levels.  The  MTF 
equations  and  plots  for  all  six  illuminations  were  retained  for  informational  purposes 
(see  Appendix  A).  The  above  metrics  were  computed  only  for  the  three  illuminations 
used  and  for  both  image  intensifier  tubes.  The  metrics  for  both  tubes  were  averaged  to 
get  an  estimate  of  the  system  metrics.  The  results  of  the  metric  calculations  can  be 
found  in  Table  3. 

Once  the  metric  values  were  determined  for  each  illumination  level,  they  were 
regressed  on  the  performance  measures  described  below  using  STATGRAPHICS 
(version  5).  The  performance  measures  for  this  study  were  taken  from  Pierce  (1994). 

RESULTS 

Once  the  data  were  collected,  the  MTFA,  ICS,  SQRI,  Resolution,  and  SNR 
metrics  were  calculated.  The  metrics  were  compared  to  the  two  visual  detection  and 
recognition  performance  measures  from  Pierce  (1994)  using  statistical  regression 
techniques.  The  basic  regression  equation  for  each  metric  is  of  the  form: 

PERFORMANCE  =  bo  +  bjM,  (8) 

where  M  denotes  the  metric  of  interest  (  MTFA,  SQRI,  ICS,  Resolution,  or  SNR), 
bo  denotes  the  intercept  of  the  regression  equation,  and  bi  denotes  the  slope  of  the 
regression  equation. 
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TABLE  3.  Results  of  Metric  Calculations 


MTFA 

ICS 

SQRI 

RES 

SNR 

75%  MOON 

A.em 

1821.33 

85.0094 

35.9 

72.40 

50%  MOON 

4.6204 

1821.88 

85.1181 

38.1 

70.95 

25%  MOON 

4.6572 

1840.84 

85.5793 

38.1 

66.60 
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In  the  following  subsections,  each  metric  is  compared  to  the  performance  measures  and 
evaluated  to  determine  the  metric  best  related  to  operator  performance.  The  statistical 
measures  of  R  ,  the  p  value  associated  with  R  ,  and  Mallow's  Cp  statistic  were  used 

to  assess  the  best  fit  models.  For  a  summary  of  the  evaluation  tool  results  see 
Appendix  B. 

MTFA  vs.  Target  Detection  Performance 

The  R  value  for  the  regression  of  MTFA  on  target  detection  is  0.5065 
(p  =0.495).  The  regression  equation  is: 

Performance  =  9.621  -  (196.6)(MTFA)  (8) 

Figure  8  shows  a  plot  of  Equation  8. 

ICS  vs.  Target  Detection  Performance 

The  R  value  for  the  regression  of  ICS  on  target  detection  is  0.9188 
(p  =  0.1839).  The  regression  equation  is: 

Performance  =  853.1-  (0.439)(ICS)  (9) 

Figure  9  shows  a  plot  of  Equation  9. 
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SORI  vs.  Target  Detection  Performance 


The  value  for  the  regression  of  SQRI  on  target  detection  is  0.9829 
(p  =0.0835).  The  regression  equation  is: 

Performance  =  1486.5  -  (i6.854)(SQRI)  (10) 

Figure  10  shows  a  plot  of  Equation  10. 

Resolution  vs.  Target  Detection  Performance 

The  R  value  for  the  regression  of  Resolution  on  target  detection  is  0.5519 
(p  =0.4669).  The  regression  equation  is: 

Performance  =  161.229  -  (2.977)(Resolution)  (11) 

Figure  1 1  shows  a  plot  of  Equation  11. 

SNR  vs.  Target  Detection  Performance 

The  R^  value  for  the  regression  of  SNR  on  target  detection  is  0.9949 
(p  =0.0453).  The  regression  equation  is: 

Performance  =  -67.802  +  (1.682)(SNR)  (12) 

Figure  12  shows  a  plot  of  Equation  12. 
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MTFA  vs.  Target  Recognition  Performance 

The  R  value  for  the  regression  of  MTFA  on  target  recognition  is  0.0165 
(p  =0,9181).  The  regression  equation  is: 

Performance  =  170.178  -  (28.784)(MTFA)  (13) 

Figure  13  shows  a  plot  of  Equation  13. 

ICS  vs.  Target  Recognition  Performance 

The  R  value  for  the  regression  of  ICS  vs.  target  recognition  is  0,3363 
(p  =0.6062).  The  regression  equation  is: 

Performance  =  431.008  -  (.2157)(ICS)  (14) 

Figure  14  shows  a  plot  of  Equation  14. 

SQRI  vs.  Target  Recognition  Performance 

The  R  value  for  the  regression  of  SQRI  on  target  recognition  is  0.4909 
(p  =0.5058).  The  regression  equation  is: 

Performance  =  860.646  -  (9.667)(SQRI)  (15) 

Figure  15  shows  a  plot  of  Equation  15, 
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Figure  13.  MTFA  recognition  regression  plot. 
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Resolution  vs.  Target  Recognition  Performance 

The  R  value  for  the  regression  of  Resolution  on  target  recognition  is  0.9951 
(p  =0.0447).  The  regression  equation  is: 

Performance  =  157.942  -  (3. 245) (Resolution)  (16) 

Figure  16  shows  a  plot  of  Equation  16. 

SNR  vs.  Target  Recognition  Performance 

The  R  value  for  the  regression  of  SNR  on  target  recognition  is  0,5509 
(p  =0.4675).  The  regression  equation  is: 

Performance  =  -34.445  +  (1.016)(SNR)  (17) 

Figure  17  shows  a  plot  of  Equation  17. 

The  Mallow’s  Cp  statistic  was  calculated  for  each  of  regression  equations.  The 
results  of  the  Cp  statistic  confirmed  the  results  of  the  other  evaluation  tools,  namely 
that  for  target  detection  the  SNR  regression  equation  has  the  least  bias  and  for  target 
recognition  the  Resolution  regression  equation  has  the  least  bias. 
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Resolution 

Figure  16.  Resolution  recognition  regression  plot. 
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In  addition  to  the  regression  equations  and  Mallow’s  Cp  statistics,  t-tests  on  the 
differences  between  R  values  and  on  observed  metric  correlations  were  computed. 

The  t-tests  found  that  there  were  no  significant  differences  among  the  metrics  for  the 
target  detection  and  target  recognition  tasks  (Table  4).  The  only  metric  correlations 
that  were  significantly  different  from  2ero  were:  SNR  for  target  detection  and 
Resolution  for  target  recognition  (Table  5). 

DISCUSSION 

The  only  metric  that  correlated  significantly  with  target  detection  performance 
was  SNR.  This  suggests  that  noise  levels  in  the  NVGs  actually  drive  operator  target 
detection  performance.  This  finding  is  consistent  with  those  of  Riegler,  Whitely,  Task, 
and  Schueren  (1991),  who  found  that  as  the  SNR  of  NVGs  improves  so  does 
performance.  Also,  Beaton  (1984)  found  SNR  to  be  the  best  performance  predictor 
when  noise  in  a  system  was  a  consideration.  This  also  makes  sense  from  a 
psychophysical,  signal-to-noise  ratio  perspective,  as  the  signal  strength  increases  it  is 
easier  to  pick  it  out  from  a  noisy  background. 

For  target  recognition.  Resolution  has  the  only  significant  R  value,  0.9951. 

One  would,  however,  expect  recognition  performance  to  improve  as  system  resolving 
power  improved,  so  the  large  R  value  does  not  come  as  a  surpnse. 
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TABLE  4.  T-test  Results  for  Differences  * 


UNFAIR 

Detection 

t  observed 

Recognition 

t  observed 

MTFA-ICS 

omi 

0.050 

MTFA-SQRI 

0.032 

0.004 

MTFA-Resolution 

0.002 

0.014 

MTFA-SNR 

0.033 

0.025 

ICS-SQRI 

0.026 

0.050 

ICS-Resolution 

0.027 

0.050 

ICS-SNR 

0.031 

0.050 

SQRI-Resolution 

0.032 

0.010 

SQRI-SNR 

0.022 

0.022 

Resolution-SNR 

0.033 

0.014 

1.  The  critical  t-test  value  was  2.132  (df  =4,  a  =0.05) 
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TABLE  5.  T-test  Results  for  Correlation  Equal  to  Zero  ^ 


r 

t  observed 

Detection 

MTFA 

0.7116 

1.01 

ICS 

0.9585 

3.36 

SQRI 

0.9914 

7.58 

Resolution 

0.7429 

1.11 

SNR 

0.9974 

13.96 

Recognition 

MTFA 

0.1283 

0.1294 

ICS 

0.5798 

0.7116 

SQRI 

0.7006 

0.9818 

Resolution 

0.9975 

19.950 

SNR 

0.7422 

1.1075 

1.  The  critical  t-test  value  was  12.71  (df  =1,  a  =0.05) 
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Another  consideration  in  regard  to  the  values  of  the  MTFA,  ICS,  and  SQRI 
metrics  is  that  system  noise  was  not  accounted  for  when  making  the  measurements. 
Beaton  (1984)  and  Barten  (1991)  found  that  system  noise  elevated  the  CTF.  The 
implication  here  is,  the  metric  values  calculated  in  this  paper  are  actually  higher  than 
they  would  be  if  noise  were  taken  into  account.  Also,  the  MTFs  did  not  vary  much 
with  changes  in  illumination  levels,  so  changes  in  performance  effects  were  not 
captured  by  the  MTFA,  ICS,  and  SQRI  metrics.  These  findings  suggest  that  the 
ANVIS-6  NVGs  are  noise-limited  systems  and  the  that  MTF-based  metrics  should  be 
corrected  for  noise  for  effective  use  in  modeling  NVGs. 

Also,  when  one  looks  at  the  plots  of  the  MTFs  on  the  same  graph  (Figure  18),  it 
can  be  noted  that  on  the  25%  moon  illumination  curve  there  appears  an  abnormal  bump 
at  2.5  cycles  per  degree.  This  bump  causes  the  MTFA,  ICS,  and  SQRI  values  to  be 
slightly  greater  for  25%  illumination  than  for  50%  and  75%  illuminations.  Since  the 
bump  occurs  at  a  low  spatial  frequency  SQRI  is  more  effected  than  MTFA  and  ICS. 
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Figure  18.  MTF  Plots. 
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CONCLUSION 


The  main  conclusions  reached  in  this  study  are:  that  the  SNR  image  quality 
metric  serves  as  the  best  predictor  of  performance  in  target  detection  tasks  and 
resolution  serves  as  the  best  performance  predictor  in  target  recognition. 

The  purpose  of  this  study  was  to  provide  an  objective  assessment  of  the  image 
quality  of  ANVIS-6  NVGs  using  five  image  quality  metrics.  This  purpose  was 
fulfilled;  although  only  one  metric  was  found  to  predict  each  performance  task.  It  is 
suggested  by  the  present  findings  that  the  MTFA,  ICS,  and  SQRI  metrics  need  to  be 
reformulated  to  account  for  the  effects  of  noise  on  visual  target  detection  and 
recognition  performance.  The  trends  shown  by  the  metrics  were  as  expected.  To  fully 
explore  the  metric  trends,  a  more  extensive  or  follow  on  study  should  be  conducted. 
The  next  study  should  include  additional  illumination  levels  to  effectively  cover  the 
operating  range  of  the  goggles.  Also,  the  study  should  take  noise  into  account  when 
calculating  MTFA,  ICS,  and  SQRI.  The  increase  in  illumination  levels  should  also 
result  m  more  statistical  power  when  analyzing  regression  equations  and  R  values, 
perhaps  resulting  in  higher  confidence  levels  for  the  metrics. 

The  trends  observed  in  the  present  finding  can  be  used  as  engineering  criteria  to 
guide  and  evaluate  NVG  system  improvements  and  to  predict  the  usability  of  these 
devices.  Future  improvements  in  SNR  and  Resolution  will  lead  to  increased  operator 
performance.  Also,  if  unit  level  tests  are  to  be  developed  to  rate  NVGs  already  in  the 
field,  the  tests  should  involve  SNR  and  Resolution  measures. 
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Appendix  A 
MTF  Equations 

k  =  7t  ~2  /  4  ln2;  x  =  spatial  frequency  in  cycles  per  degree. 

MTF  at  100%  moon  illumination 

y  =9.671E-2  +  8.252  E-1*  Exp(k*  1.021E-17*x^) 

=  .9928 

MTF  at  75  %  moon  illumination 

y  =1.061E-1  +  8.1814  E-1*  Exp(k*  1.184E-1%‘2) 

=  .9975 

MTF  at  50%  moon  illumination 

y  =i.l011E-l  -I-  8.284  E-1  *Exp(k*  1.177E-17*x^) 
R^  =  .9978 

MTF  at  25%  moon  illumination 

y  =9.900E-2  +  8.409  E-l*Exp(k*  1.170E-l^*x^) 
R^  =  .9993 

MTF  at  10%  moon  illumination 


y  =9.589E-2  -t-  9.143  E-1*  Exp(k*  1.144E-l^*x^) 
R^  =  .9996 


MTF  at  1  %  moon  illumination 


y  =9.229E-2  +  8.298  E-1*  Exp(k*  1.192E-l'2*x7) 
R^  =  .9978 
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Appendix  B 


METRIC  EVALUATION  RESULTS 


DETECTION 

- ? - 

P 

Mallow  Cp 

MTFA 

0.5065 

0.4958 

1.9870 

ICS 

0.9188 

0.1839 

1.1624 

SQRI 

0.9829 

0.0835 

1.0343 

RESOLUTION 

0.5519 

0.4669 

1.8963 

SNR 

0.9949 

0.0453 

1.0101 

RECOGNITION 

? - 

P 

Mallow  Cp 

MTFA 

0.0165 

0.9181 

2.9670 

ICS 

0.3363 

0.6062 

2.3275 

SQRI 

0.4909 

0.5058 

2.0182 

RESOLUTION 

0.9951 

0.0447 

1.0098 

SNR 

0.5509 

0.4675 

1.8982 
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Appendix  C 


Modulation  Transfer  Factors 


MIFs 

100% 

Moon 

75% 

Moon 

50% 

Moon 

25% 

Moon 

10% 

Moon 

1% 

Moon 

1.0  cy/deg 

0.863 

0.899 

0.903 

0.899 

0.967 

0.895 

2.5  cy/deg 

0.798 

0.682 

0.689 

0.722 

0.783 

0.675 

5.0cy/deg 

0.400 

0.354 

0.354 

0.346 

0.379 

0.339 

10  cy/deg 

0.130 

0.123 

0.119 

0.119 

0.115 

0.108 

15cy/deg 

0.090 

0.092 

0,087 

0.087 

0.087 

0.079 
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Appendix  D 

Signal-to-Noise  Ratio  Data 


100% 

Moon 

75% 

Moon 

50% 

Moon 

25% 

Moon 

10% 

Moon 

1% 

Moon 

Left  Tube 

702 

699 

697 

688 

628 

460 

Signal  (mV) 

Left  Tube 

10.1 

9.9 
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